A Review of Bayesian Neural

MacKay's Bayesian framework for backpropagation is a practical and powerful means to improve the generalisation ability of neural networks. It is based on a Gaussian approximation to the posterior weight distribution. The framework is extended, reviewed and demonstrated in a pedagogical way. The notation is simpliied using the ordinary weight decay parameter, and a detailed and explicit procedure for adjusting several weight decay parameters is given. Bayesian backprop is applied in the prediction of fat content in minced meat from near infrared spectra. It outperforms \early stopping" as well as quadratic regression. The evidence of a committee of diierently trained networks is computed, and the corresponding improved generalisation is veriied. The error bars on the predictions of the fat content are computed. There are three contributors: The random noise, the uncertainty in the weights, and the deviation among the committee members. The Bayesian framework is compared to Moody's GPE. Finally, MacKay and Neal's Automatic Relevance Determination, in which the weight decay parameters depend on the input number, is applied to the data with improved results.